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INTRODUCTION 

After spending decades introducing curriculum and instructional programs with little to 
show in the way of student achievement results, the District of Columbia Public Schools 
(DCPS), under the leadership of Chancellor Michelle Rhee, decided to shift its focus. Empha- 
sizing what to teach had yielded few results. Dogged by its standing as perhaps the poor- 
est performing urban school system in the country, DCPS decided to address head-on the 
issue of how to teach. The goal was straightforward: to ensure instructional excellence in 
every classroom. The system set out to achieve this goal by defining the elements of effective 
instruction, creating a common understanding of those elements system-wide, and develop- 
ing an accountability and support system aligned to these elements. This work represented a 
profound shift for a system whose hallmarks had been teacher autonomy and isolation and 
the use of instructional materials as the default curriculum. 

DCPS went about this effort in a deliberate and systematic way. During the 2008-09 school 
year, the system developed its Teaching and Learning Framework, a set of standards for 
instructional practice and an aligned rubric that would guide every teacher in the system, 
regardless of grade or subject taught. By September 2009, teachers, principals and central 
office staff were introduced to the framework. The 2009-10 school year was spent train- 
ing teachers on the standards and unveiling a new teacher performance evaluation system 
aligned to them. The district then undertook a careful review of the framework, the evalua- 
tions, and their implementation, and in the 2010-11 school year made substantial modifica- 
tions. 

For district leaders, linking the standards to accountability was key. By doing so, the district 
ensured everyone would take notice. It also ran the risk that the anxiety provoked by the 
new high-stakes accountability system would eclipse teachers’ deep understanding of the 
standards and ownership of them. This was a risk district leaders were willing to take. Defin- 
ing instructional excellence and organizing support and accountability on this definition is 
essential to driving systemic instructional improvement. For many school systems this work 
is profoundly counter-cultural and requires a fundamental shift in how they think about their 
role and execute it. Organizing support and accountability around teaching expectations 
allows systems to align themselves to drive instructional improvement in ways that create 
tremendous coherence for teachers and schools and maximize the likelihood of powerful 
outcomes for students. DCPS has taken a comprehensive approach to this work and there is 
much to be learned from the work in progress. 

This profile tells the story of DCPS’s development and implementation of teaching standards, 
an aligned evaluation system, and a new pay-for-performance compensation system. The 
five critical lessons learned from DCPS’s work that are explored in detail in the profile are: 
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Tap the potential of teaching standards to create common expecta- 
tions and language about instruction and to focus everyone in the 
system on thinking about how teachers teach and why. 

Anticipate that the hardest and most important part of a teacher 
performance management system is helping teachers improve their 
practice. 

Consider the trade-offs associated with introducing teaching 
standards and an aligned evaluation system simultaneously, as 
compared to sequentially. 

Understand that building a robust teacher performance manage- 
ment system is going to require the continuous development of 
organization capacity. 



Think strategically about organizing the system to drive the design 
and implementation of teaching standards, the associated evalua- 
tion system, and the related supports. 



While DCPS is in many ways unique, these learnings can apply to any district that is making 
an effort to establish standards and aligned evaluation and compensation systems. They 
provide important guidance, particularly for well-established bureaucratic systems, about 
how to approach this work in a way that drives both instructional excellence and a retooling 
of the central office to support instructional improvement, and ultimately student learning. 
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BACKGROUND 



When Michelle Rhee took the job of chancellor of the DCPS in June 
2007, there was little good news to report. Students in DCPS scored 
at the bottom on the National Assessment of Educational Progress 
(NAEP), compared to 11 other large, urban school systems, includ- 
ing New York, Boston, Atlanta, Cleveland, Miami, and Chicago. The 
level of poverty in the District of Columbia was not an explana- 
tion; when the performance of DC students who lived in poverty 
was compared with similar students in the other school systems, 
the results were equally discouraging. 

White students — who made up 16 percent of the DCPS student 
body and tended to be relatively affluent and concentrated in the 
schools in the Northwest corner of the district — were the only 
group that scored well. As a result, DCPS had the widest racial 
achievement gap of any of the 11 urban districts. 



Percent of Students Scoring Basic or 
Above on the National Assessment 
of Educational Progress (NAEP) 



STUDENT 

POPULATION 


AVERAGE 
OF 11 LARGE, 
URBAN SCHOOL 
SYSTEMS 


DC 

AVERAGE 


4th Graders 
with Basic Skills 


67% 


38% 


Middle Schoolers 
with Basic Skills 


51% 


26% 







DCPS achieved these dismal results despite the fact that they spent 
more than most other districts. DCPS spent $12,979 per pupil each 
year, which made it the third highest spending district out of the 
100 largest school systems in the country. Despite this high level 
of spending, DCPS ranked dead last in spending on teachers and 
instruction. Thus DCPS was spending an enormous amount on 
things that did not produce results. And the lack of results had a 
substantial cost. 

Over the past decade, there was a tremendous flight out of the 
system: a 40 percent decline in student enrollment, from 76,000 
students in 1997 to 46,000 in 2007. Many of these students fled 
to the burgeoning charter school sector, one of the largest in the 
nation. Thirty-eight percent of DCPS’s school-aged children attend 
charter schools. 1 

Adrian Fenty, DC’s recently elected mayor, recruited Rhee for the 
job of schools chancellor in the spring of 2007. He intended to 
make education his top priority and soon after his election gained 
the authority to appoint a chancellor who reported directly to him. 

Rhee, who started her career as a Teach for America corps member 
and founded The New Teacher Project, was perceived as an unusu- 
al choice, given her lack of school system leadership experience. 

Fenty was drawn by her reputation as a smart and fearless reformer. In recruiting Rhee, Fenty 
promised his full support of her aggressive agenda. 

From the start, Rhee made clear her plans for a complete transformation of the system. Her 
moves were ambitious and unequivocal. By June 2008 she had closed 23 schools (15 percent 
of the whole system). By the fall of that year she had placed 26 additional schools under 
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DCPS DEMOGRAPHICS: 2010 


Number of Schools 


123 


Number of Teachers 


3,510 


Number of Students 46,515 


Experience Level of Teachers: 


One to two years 


22% 


Three to five years 


15% 


Six to ten years 


30% 


Ten to twenty years 


14% 


More than twenty years 


20% 


Diversity of Students: 




Black 


69% 


Hispanic 


13% 


Asian 


2% 


White 


16% 


Students Eligible for Free 


or Reduced Price Lunch 


62% 





1 Deborah Simmons, "DC 
Charter Schools Face Unfunded 
Mandates, "The Washington 
Times, August 22, 2010, 
http://www.washingtontimes. 
com/news/2010/aug/22/dc- 
charter-schools-face-unfunded 
-mandates/ 



MARCH 2011 



3 




THE ASPEN INSTITUTE 



restructuring based on their performance. In her first two years on the job Rhee hired 77 new 
principals (60 percent of the principal workforce). All principals received increased author- 
ity in staffing and high-performing schools earned additional autonomies. Rhee quadrupled 
the investment in teacher and principal professional development and realized a 30 percent 
increase in the percent of classes taught by “highly qualified teachers” (under the federal No 
Child Left Behind Act). In 2009, the percent of teachers eligible for tenure who received it was 
75 percent, down from 99 percent in 2008. 



Changes in the central office were equally 
dramatic. All non-union, central office jobs 
were re-categorized as “at will” positions, and 
92 central office staff were terminated. Rhee 
established the first central office performance 
review in more than two decades and intro- 
duced scorecards for each central office depart- 
ment. She introduced and led School Stat, a 
weekly performance management session with 
her senior staff, in which performance metrics 
and accountability for individual departments 
were closely analyzed and discussed. 

In Rhee’s first two years student results 
improved substantially. Student performance 
levels on the DC-CAS, the district’s assessment 
system, trended upwards, with a clear spike 
in 2008. DC’s performance on the 2009 NAEP 
showed steady improvement. When compared 
with like urban districts in the NAEPTrial Urban 
Districts Assessment (TUDA), DC had the great- 
est gains of any district in fourth grade math and had moved from the bottom of the pack to 
rank in the top five in fourth and eighth grade math performance. As with any transition in 
leadership it is not clear how much of the credit for these improvements can be attributed to 
Rhee’s administration and how much is the result other predecessor’s efforts. Yet these gains 
in student achievement were promising and challenged DCPS to maintain strong, continued 
growth. To accomplish this, the system turned its attention to the quality of instruction in 
classrooms. 



Students Scoring Proficient or Above on the DC 
Comprehensive Assessment System (DCCAS) 

■ Elementary Reading ■ Elementary Math ■ Secondary Reading Secondary Math 




4 



DCPSTLF AND IMPACT: EXPECTATIONS, ACCOUNTABILITY, AND SUPPORT 



THE ASPEN ) INSTITUTE 



EDUCATION & SOCIETY PROGRAM 



Teaching and Learning Infrastructure 

As a first step in implementing her instructional improve- 
ment agenda, Rhee and her leadership team introduced the 
Effective Schools Framework in 2008-09. This framework 
articulates the foundational elements of effective schools and 
guides central office support and school improvement efforts. 
The framework, which is aligned to the district five-year stra- 
tegic plan, includes the following elements: 

Teaching and Learning:Teachers plan and deliver 
high-quality, rigorous, standards-based instruction 

Leadership: Principals are instructional leaders 

Job-Embedded Professional Development: High- 
quality professional development is job-embedded, 
aligned to district and local school goals, data- 
driven, and differentiated 

Resources: Resources are allocated to drive instruc- 
tional improvement and student achievement 

Safe and Effective Learning Environment: Policies, 
procedures and practices support a safe environ- 
ment characterized by high expectations, mutual 
respect, and focus on teaching and learning 

6 Family and Community Engagement: Schools invite 
and engage family and community members as 
partners in the critical work of creating effective 
learners and schools 

The Plan-Do-Study-Act inquiry cycle is woven through all six 
components of the framework to reflect the system’s commit- 
ment to data-driven inquiry and decision-making. 

Principals’ training in school year 2009 focused heavily on 
building a common understanding of the Effective Schools 
Framework and its implications for principal practice. 



DC NAEP Performance Compared to 
Districts Participating in Trial Urban 
District Assessment (TUDA) That 
Realized the Greatest Gains 



MATH 




GRADE 4 


1 




2003 


2007 


2009 


National Average 


234 


239 


239 


Large City Average 


224 


230 


231 


Atlanta 


216 


224 


225 


Boston 


220 


233 


236 


DC 


205 


214 


220 


NYC 


226 


236 


237 


San Diego 


226 


234 


236 


MATH 




GRADE 8 






2003 


2007 


2009 


National Average 


276 


280 


282 


Large City Average 


262 


269 


271 


Atlanta 


244 


256 


259 


Boston 


262 


276 


279 


DC 


243 


248 


251 


Houston 


264 


273 


277 


LA 


245 


257 


258 


San Diego 


264 


272 


280 


READING 




GRADE 4 






2003 


2007 


2009 


National Average 


216 


220 


220 


Large City Average 


204 


208 


210 


Atlanta 


197 


207 


209 


Boston 


206 


210 


215 


DC 


188 


197 


203 


NYC 


210 


213 


217 


READING 




GRADE 8 






2003 


2007 


2009 


National Average 


261 


261 


262 


Large City Average 


250 


250 


252 


Atlanta 


240 


245 


250 


DC 


240 


241 


240 


Houston 


246 


252 


252 


LA 


234 


240 


244 
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DCPS Effective Schools Framework 




At the heart of the framework is teaching and learning. To underscore that 
emphasis, in 2008-09 the district introduced instructional coaching 
and hired 173 coaches. Every school had at least one coach; larger 



schools received two. In the first year, there was wide variation in 
what coaches did in schools. Their role was to provide profession- 
al development and classroom-based support in response to 



teachers’ needs, but there was not a system-wide understand- 
ing of what this meant or looked like. Coaches were often seen 
as an extra person who could help the school with whatever 
was most pressing. The lack of both a clear understanding of 
the work of coaches and a system-wide instructional improve- 



ment strategy on which they could organize their work led to 
inconsistent and diffused coaching efforts and an unclear return 
on the investment. 



Teaching and Learning Framework 2 



Having clarified the expectations of schools through the Effective Schools 



Framework and having begun to approach instructional improvement through coach- 
ing, senior leaders knew the next move was to define instructional expectations. Doing so 
would provide focus for coaching and professional development and would address the most 
important school-related variable affecting student achievement: instruction. The question 
was; what would be the most effective way to get the entire organization focused on instruc- 
tion? Michael Moody, Chief Academic Advisor to the Chancellor, described the opportunity 
and challenge this way: “We needed to create something compelling that people could really 
rally around [related to teaching and learning]. Jason Kamras, Director of Teacher Human 
Capital at the time, was working on developing the teacher evaluation system, which created 
an opportunity to engage around instruction.”The plans for a new evaluation system served 
as the catalyst for defining teaching standards. 

As a first step toward that end, in fall 2008 DCPS embarked on an initiative to define effec- 
tive instruction. Given a long history of instructional materials implementation efforts, this 
was a dramatic shift for the district. Instead of emphasizing what teachers teach, they set 
out to provide much more explicit guidance about how to teach and the why behind specific 
pedagogical strategies. Moody explained that, by focusing on teaching, the system intended 
to “empower teachers to take control of and responsibility for what they’re teaching. In Year 
One we’re focusing entirely on pedagogy; there’s no focus on content intentionally. Next year 
we’ll weave some content back in, [to address the particulars of] howyou use the framework 
in [say,] social studies class. Historically, the district has been very good at saying ‘Do this,’ 
but it never stuck. We’re focusing on the why over the what.” Focusing on empowering teach- 
ers and holding them responsible for improving instruction was an intentional strategy to 
disrupt the prevailing teacher culture that was simultaneously disempowering and lacking 



2 The framework presented in 
this section is the revised 2.0 
version that was developed 



accountability. 



in the summer of 2010; it 
reflects revisions made based 



on learning in year one. 
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Shifting the focus to instruction required reorganizing the system from the central office to 
the classroom. Under the old paradigm, which emphasized curriculum, the curriculum and 
instruction department had grown large and had a very "content-specific point of view,” 
according to Carey Wright, who served as the Deputy Chief of Teaching and Learning during 
the rollout of IMPACT and is now the Chief Academic Officer. There was little coherence or 
collaboration across the different disciplines. As the framework was developed, the curricu- 
lum and instruction department was downsized, leaving a core staff of eight to ten people 
whose roles were redefined as generalists. By focusing on effective instructional practices 
that cut across content areas, this initiative would touch every teacher in the system and 
would provide a foundation of instructional expectations and coherence on which profes- 
sional development and accountability could be organized. 



DC PS Teaching and Learning Framework 2.0 

DCPS contracted with Insight Education Group, an educational consulting 
firm, to draft the framework. The development process was iterative, 
driven by a small, core team of six people from the central office. 
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Early in the process the team reviewed more than a dozen 
frameworks (e.g., Marzano, CLASS, Danielson, Teaching as 
Leadership), analyzed their commonalities and differences, 
and discerned what elements would be most important 
to DCPS. The team agreed on the three big elements of 
the framework - Plan, which encompasses what teach- J 
ers do to plan instruction and the learning environ- J 
ment; Teach, which represents their actual instruction- £ 
al practices; and Increase Effectiveness, which includes § 

how they assess student learning, track progress, and 1 

use the data to inform their (re)teaching (See Appendix 
Aforfull-size image of the teaching cycle). 



# 



& 



PLAN 



TEACH 



% 



% 



^ A 



INSTRUCTION 

1. Develop annual student 
achievement goals 

2. Create standards-based unit plans 
and assessments 

3. Create objective-driven lesson plans 

LEARNING ENVIRONMENT 

4. Adopt a classroom behavior 
management system 

5. Develop classroom procedures 
and routines 

6. Organize classroom space 
and materials 



1. Lead well-organized, objective- 
driven lessons 

2. Explain content clearly 

3. Engage students at all learning levels 
in rigorous work 

4. Provide students multiple ways to engage 
with content 

5. Check for student understanding 

6. Respond to student misunderstandings 

7. Develop higher-level understanding through 
effective questioning 

8. Maximize instructional time 

9. Build a supportive, learning-focused classroom 
communtiy 



% 




With these three elements agreed upon, the team chose to 
focus first on Teach, the heart of the framework, and iden- 
tified nine sub-elements of that element. Once those were 
defined, the team shared the draft with principals and teachers 
to get feedback. To solicit teachers’ input, the team held a series of 
teacher focus groups across the city. To get comments from principals, 
the team made the framework the focus of several of the monthly Principals 
Academies. Central office leaders also reviewed the framework three or four different 
times during the development process and the team reached out to experts in instructional 
excellence and teacher evaluation to get their reactions to the framework and advice about 
revisions. The feedback informed refinements but did not lead to fundamental changes in 
the three main elements or the nine sub-elements. However, district leaders made clear that 
they would reviewthe implementation of the framework carefully and revise the framework 
as needed, introducing a version 2.0 of the framework for use in the 2010-11 school year. 

To better understand the choices DCPS made in developing its framework, it is worth 
comparing it to another commonly used framework to see the similarities and differences 



INCREASE 

EFFECTIVENESS 



1. Assess student progress 

2. Track student progress data 

3. Improve practice and re-teach in response to data 
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DC PS Teaching and Learning Framework 2.0 



PLAN 

Instruction 

1. Develop annual student 
achievement goals 

2. Create standards-based unit 
plans and assessments 

3. Create objective-driven daily 
lessons 

Learning Environment 

4. Adopt a classroom behavior 
management system 

5. Develop classroom procedures 
and routines 

6. Organize classroom space and 
materials 



TEACH 

1. Lead well-organized, objective- 
driven lessons 

2. Explain content clearly 

3. Engage students at all learning 
levels in rigorous work 

4. Provide students multiple ways 
to engage with content 

5. Check for student understanding 

6. Respond to student misunder- 
standings 

7. Develop higher-level understand- 
ing through effective questioning 

8. Maximize instructional time 

9. Build a supportive, learning- 
focused classroom community 



INCREASE EFFECTIVENESS 

1. Assess student progress 

2. Track student progress data 

3. Improve practice and re-teach in 
response to data 



(See Appendix B for excerpts from version 2.0 of the framework). The Danielson frame- 
work, which a number of school systems around the country use, provides an interesting 
comparison. Both frameworks emphasize planning, learning environment, and instruction. 
But they differ in what else they emphasize and the level of explicitness reflected. For exam- 
ple, "Increase Effectiveness,” which focuses on assessing and tracking student progress and 
using that information to inform instruction, is prominent in the DCPS framework; indeed, 
it is one of the three big elements. The Danielson framework reflects similar expectations 
regarding the use of assessments, but they are reflected in three components: “planning and 
preparation,” "instruction,” and "professional responsibilities.” Another example relates to 
how “professional responsibilities” are addressed. This domain includes: reflection on teach- 
ing, record keeping, communication with families, participating in professional community, 
professional growth and development, and professionalism. DCPS addresses most of these 
ideas in its measure of “core professionalism” and “commitment to the school community,” 
two elements of its overall teacher evaluation, but chose not to include it in the framework, 
which focuses tightly on the teaching and learning cycle, the work that directly facilitates 
student learning. The table below shows the broad categories of each framework. 



DANIELSON’S FRAMEWORK 
FORTEACHING DOMAINS 


DCPS’S TEACHING AND 
LEARNING FRAMEWORK 


Planning and Preparation 


Plan (Instruction and Learning Environment) 


The Classroom Environment 


Teach 


Instruction 


Increase Effectiveness 


Professional Responsibilities 
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A closer look at the two frameworks reveals another difference, related to the level of detail 
and explicitness. To illustrate that difference, the table below compares the components 
under each framework’s element focused on instruction and shows the extent to which 
DCPS parsed the elements of instruction, compared with Danielson. DCPS’s rubric devotes 
two sub-elements to student engagement, focusing on rigor and multiple ways of engaging 
with content, while the Danielson rubric has one component related to engagement. Simi- 
larly, DCPS uses three components to differentiate among student understanding, misunder- 
standing, and higher-level understanding. The language of the Danielson rubric appears to 
be more holistic. 



DANIELSON DOMAIN 3: INSTRUCTION 


DCPS FRAMEWORK: TEACH 


1. Communicating with students 


1. Lead well-organized, objective-driven lessons 


2. Using questioning and discussion 
techniques 


2. Explain content clearly 


3. Engaging students in learning 


3. Engage students at all learning levels in rigorous work 


4. Using assessment in instruction 


4. Provide student multiple ways to engage with content 


5. Demonstrating flexibility and 
responsiveness 


5. Checkfor student understanding 




6. Respond to student misunderstanding 




7. Develop higher-level understandingthrough effective questioning 




8. Maximize instructional time 




9. Build a supportive, learning-focused classroom community 



Implementing the Framework 

With the framework developed by June 2009, the looming question was how to roll it out 
to the entire system at the start of the 2009-10 school year. How could the system build the 
capacity required to introduce the framework to more than 3,500 teachers and hundreds of 
school administrators and central office staff and prepare all of them to use it to guide their 
instruction, classroom observation, and support? DCPS decided that in the first year it would 
roll out just one-Teach -of the three elements of the framework. 

Organizational capacity and the prioritization of instruction drove this decision. The frame- 
work reflected the system’s first effort in decades to develop consistent instructional practic- 
es system-wide. Spending a full year building deep understanding of Teach, the element that 
addresses instructional practices and teacher-student interactions, felt essential to building 
a solid foundation of instructional expectations, consistency in instructional practices, and a 
culture focused on instructional quality across the district. At the same time, district leaders 
knew that the three elements of the framework— Plan, Teach, Improve Effectiveness— are 
inextricably linked and can’t be neatly separated. Focusing on Teach would surely touch on 
Plan and Improve Effectiveness. 
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3 These days were created 
through negotiations with the 
mayor and in close partner- 
ship with other city agencies, 
to provide programming for 
students while they would not 
be in school. 


Professional Development 

Before the ink had dried on the framework, the professional development division of the 
system began developing three-hour training modules for each of the nine sub-elements 
of Teach. The professional development department’s staff of 14 full-time specialists wrote 
these modules and prepared to teach them. DCPS developed a train-the-trainer model to 
build the capacity required to train every teacher in the system in fall 2009. Each school was 
asked to identify a person on the staff who would serve as the school’s lead on the frame- 
work and who would be trained in the Teach modules. That person would then be expect- 
ed to teach the module to his or her colleagues back at school. Sixty people (80 percent of 
whom were instructional coaches; 20 percent of whom were principals or assistant princi- 
pals) participated in three days of introductory training on the framework. Then each trainer 
selected certain modules from Teach that they wanted to be able to train others on. For each 
of these elements they participated in an in-depth training that was competency-based, 
requiring them to pass a performance assessment for any module they would teach. 

The system-wide training began with a three-day introduction to the framework for princi- 
pals in August. The trainers participated in this training, working side-by-side with the prin- 
cipals at whose schools they would be training (most schools were paired up for the teacher 
training). The principals and trainers were expected to replicate the introductory framework 
training they were receiving. They worked in teams to build their school training agenda as 
they went through the training themselves. The system provided a flash drive of PowerPoint 
decks used in the principals’ training to each team to be used during the training in an effort 
both to be supportive and to ensure clarity of messaging. Then, the principals and trainers 
returned to their schools and replicated this training during the teacher professional devel- 
opment that preceded the start of the 2009-10 school year. 

DCPS devoted four additional contractual professional development days during the school 
year to a deeper exploration of specific elements of Teach, with the training provided by 
the system’s trainers 3 . Additionally, some schools devoted the contractual early-morning, 
30-minute daily professional development time to the framework. DCPS supplemented 
these opportunities by offering voluntary after-school and full-release-day trainings focused 
on Teach. Many schools also have common planning time, but it is unclear how much of it 
was devoted to the framework. 

Ongoing principal professional development during the 2009-10 school year was similarly 
devoted to the framework. Part of each of the monthly, full day, Principals Academy sessions 
focused on Teach. Much of this time was devoted to learning what the sub-elements of Teach 
look like in practice. Principals spent hours observing videos of teaching prepared by DCPS, 
looking for evidence of the different sub-elements of Teach and debating the extent to which 
they represented effective practice. However, there was little time devoted to teach princi- 
pals how to talk with teachers about their observations in ways that support improvement 
and how to effectively support teachers to improve. 
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Coaching 

Because of the commitment made to coaching in the 2008-09 school year, the system had 
173 instructional coaches who were positioned to play a central role in supporting imple- 
mentation of the framework at each school. Coach professional development forthe 2009-10 
school year focused on deepening coaches’ understanding of the framework and the Teach 
element, and how they could use it to guide their work with teachers. Close to 50 of the 
coaches participated in the intensive train-the-trainer model; they were among the people 
in the system who knew the most about the framework. To ensure all coaches had a clear 
understanding of the framework, much of their monthly meetings were devoted to it. Similar 
to principal training, coach professional development included lots of video observation of 
teaching to discern the elements of Teach in action. 

Coach training also focused on building skills in talking with teachers about their practice in 
ways that use observational evidence and focus on developing teacher capacity. The frame- 
work provided the foundation for coaching that the system had been missing. Yet the varied 
skill level of coaches - and the reality that coaches were a mere step ahead of teachers in 
their understanding of the framework - led to significant variability in coaches’ effectiveness 
in driving implementation of the framework. In addition to the coaches, DC hired 22 mentors 
to support the 900 new teachers hired for the 2009-10 school year. They were trained along 
with the coaches on the framework and Teach so that they could align new teacher induction 
to them. 



IMPACT: The Marriage of Support and Accountability 

To ensure that there were teeth in the Teaching and Learning Framework, DCPS deliberate- 
ly made it the centerpiece of DCPS’s new teacher evaluation system. Both the framework 
and the new teacher evaluation system were prioritized for rollout in 2009-10 school year 
and were conceived as strategic partners in a marriage of support and accountability, held 
together by clear expectations. The impetus behind revamping DCPS’s teacher evaluation 
system was predictable. Like those of many school systems, DCPS’s evaluation process was 
broken. There was a process in place on paper that included clear standards and the expecta- 



IMPACT CRITERIA 



1. Individual Value-Added: sophisticated statistical measure of teacher’s impact on student test-score gains 

2. Non-Value-Added Student Achievement Growth: non-DC CAS-based measure of teacher’s impact 
on student learning 

3. Teaching and Learning Framework: measure of instructional expertise 

4. Commitment to School Community: measure of the extent to which teacher supports colleagues 
and school's local initiatives 

5. School Value-Added: sophisticated statistical measure of your school’s impact on student achievement 

6. Core Professionalism: measures four basic professional requirements - attendance, on-time arrival, 
compliance with policies and procedures, respectful interactions with students and adults 
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4 While the focus of this paper 
is on teachers, IMPACT was 
designed to serve all school- 
based personnel including 
librarians, counselors, 
coaches, instructional para- 
professionals, and custodians, 
making it a comprehensive 
school-based performance 
management system. 


tion that teachers would be observed once a year and evaluated. In reality, there was little 
consistency in how and the extent to which this was done. 

When the ratings teachers received were juxtaposed with student achievement, the serious- 
ness of the evaluation problem crystallized. While fewer than 50 percent of students demon- 
strated proficiency on DC-CAS, 95 percent of teachers met or exceeded standards. Clearly 
many teachers earned high ratings even though their students were not performing well. 
DCPS wanted to introduce an evaluation system in which teacher performance was more 
closely tied to student achievement. Making that connection would improve both instruc- 
tional quality and student achievement by identifying high performers who could serve as 
mentors and directing resources to teachers who need them most. Kamras, then the teacher 
human capital director who now oversees all human capital for DCPS, described the initial 
goals for the new teacher evaluation system as follows: to “identify the highest performing 
teachers and leverage their talents to improve instruction system-wide and to identify and 
focus support on the lowest performing teachers and be able to release them if they don’t 
improve.”The simple premise was that, "If we can get the tails of the performance curve right 
we will provoke improvement.” 

The evaluation system, known as IMPACT, was developed during the 2008-09 school year, at 
the same time as the teaching and learning framework was developed 4 . A small team from 
the newly established Human Capital division ofthe school system tookthe lead on develop- 
ing IMPACT. This team worked closely with the team that was developing the Teaching and 
Learning Framework and, in fact, there was overlap in the membership ofthe two teams. This 
was essential because the team needed to develop a rubric for the Teaching and Learning 
Framework that would be an integral component of IMPACT. 

The IMPACT team held 50 to 60 focus groups across the district byjob category to gather input 
from teachers and other school-based staff to inform the design of IMPACT. These sessions 
addressed questions such as: "What should you be evaluated on? What measures should 
we use? How should we weigh things?” A lot of valuable ideas were collected, and Kamras 
reported, "There wasn’t universal agreement.” It was the IMPACT team’s job to synthesize the 
system’s goals, learnings from the best practices in teacher evaluation nationally, and input 
from staff to create a powerful evaluation system. During this process, DCPS communicated 
with the teachers’ union, but the district was not contractually obligated to consult with the 
union on what it was developing. 

The goals ofthe IMPACT system are to: 

outline clear performance expectations; 

provide clear feedback on teacher performance; and 

ensure every teacher has a growth plan and is getting guidance on how to increase 
effectiveness. 

To achieve these goals IMPACT was designed to measure teacher effectiveness on a set 
of criteria. The choice of criteria and the weight of each differ for teachers depending on 
whether they teach in grades and subjects that are part ofthe DC-CAS. For teachers who 
teach in DC-CAS grades and subjects, 50 percent of their evaluation rating is based on 
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“value-added” scores on DC-CAS and another 40 percent 
is based on classroom observations rated on the Teaching 
and Learning Framework rubric. For teachers for whom 
there currently are no value-added data, classroom 
observations rated on the Teaching and Learning Frame- 
work rubric account for 80 percent of their evaluation 
rating. The other criteria used to get a total IMPACT score 
include: Commitment to School Community (5 percent for 
all teachers), School Value-Added (5 percent for all teach- 
ers), and Non-Value-Added Student Achievement Growth 
(10 percent for teachers for whom value-added data are 
not available). Core professionalism is not assigned a 
percentage value, but low performance on this criterion 
can negatively affect teachers’ overall score. 



Weight of IMPACT Components 



Implementation of IMPACT 

The rollout of IMPACT tracked the rollout of the Teaching 
and Learning Framework. While the training introducing 
the framework, described earlier, was intended to build 
awareness of the framework, the fact that the system 
was also introducing a rubric to assess teacher perfor- 
mance relative to the framework influenced the training. 

Having the rubric in place by the beginning of the year 
allowed the training to focus not just on introducing the 

framework but also on distinguishing between what 

high and low levels of implementation of framework 
elements look like. Because IMPACT reflects an entirely new 

level of teacher accountability in DCPS, the rollout of both the framework and the rubric and 
its use had to include intensive support. DCPS tailored that support by creating a new posi- 
tion, Master Educators, and intensively calibrating observations and scoring. 



IMPACT 

COMPONENTS 


TEACHER WITH 
“VALUE-ADDED” 
DATA 


TEACHER 
WITHOUT 
“VALUE- 
ADDED” DATA 


Individual Value-Added 


50% 


N/A 


Non-Value-Added 


Student Achievement 
Growth 


N/A 


10 % | 


Teaching and 


Learning Framework 


40% 


80% 


Commitment to 


School Community 


5% 


5% 


School Value-Added 


5% 


5% 


Core Professionalism 


Assessed on a l-to-3 rubric. 

A score of less than 3 can 
result in a downgrading of the 
total score of the four other 
components. 









Master Educators 

Through IMPACT, teachers are evaluated annually. Every teacher in the system is observed 
five times a year for 30 minutes each time. In SY 2009-10, principals conducted three of the 
observations (one announced) and Master Educators (MEs) conducted the other two obser- 
vations (one announced). 5 The role of ME is central to IMPACT, creating the opportunity for 
highly effective teachers, coaches, and administrators to serve as the second observer and 
evaluator of teachers and ensuring more than one evaluator assesses the performance of 
each teacher. MEs are responsible for observing, analyzing, evaluating, and debriefing class- 
room observations with teachers. 



MEs conduct a minimum often classroom observations a week, which are complemented by 
an analysis of their observations, scoring, write-ups of their findings, and conferences with 
teachers to discuss their observations. Some MEs spend additional time collecting instruc- 
tional resources for teachers to share with them during their post-observation conferences. 



5 ln School Year 2010-11 one 
of the principal observations 
is announced. All of the Master 
Educator observations are 
unannounced. 
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Building relationships is an essential job skill for MEs, since they are constantly meeting new 
teachers with whom they must quickly establish trust. Such trust is essential to make the 
teacher feel at ease about having them in their classroom observing and to facilitate a post- 
observation conference that supports the teacher’s growth and development. 

Given what an important role MEs play in IMPACT, DCPS recruited, screened, and inducted 
them carefully. The system recruited nationallyforthe position, reachingouttoteacher award 
winners nationally (e.g., Teachers of the Year) and through a variety of professional networks. 
By early June 2009, there were more than 800 applicants for the 32 positions. Using a four- 
stage screening process, the district identified a group of MEs who possessed expertise as 
varied as early childhood, elementary regular education, primary literacy, secondary special 
education, and secondary math and science. Twenty-eight of the 32 (87 percent) were hired 

MEs participated in a six-week orientation, which focused on 
making them experts in the Teaching and Learning Framework 
they would use to guide their observations and on building their 
skills of observing instruction and talking and writing about 
their observations. Given that MEs would score teachers using 
the rubric and those scores would be part of a teachers’ over- 
all performance rating, consistency in ratings across MEs was 
critical to the integrity of IMPACT. For this reason MEs spent 
hundreds of hours reviewing videos of teaching, talking about 
and normingtheir ratings, and strategizing and role-playingthe 
debriefing sessions they would facilitate with teachers. 

From the beginning, MEs have been important partners with 
the system in implementing the Teaching and Learning Frame- 
work and IMPACT. Significantly, they have helped DCPS refine 
the rubric. After just a couple of weeks of practice using the 
rubric, MEs identified language that made it hard to use. They 
quickly raised their concerns to the TLF and IMPACT teams and 
produced changes. For example, MEs found that element T6 
— “fewer than 5/10... minutes of instruction time wasted”— 
was difficult to measure, even using stopwatches. In its place, they proposed "teacher is 
very effective/mostly effective/somewhat effective/ineffective at maximizing instructional 
time." Another example of their effect on the process is their recommendation to eliminate 
the descriptor in T1 that stated, “Four/Three/Two/One out of four students surveyed can 
articulate the objective of the lesson.” After trying to apply this criterion, MEs realized it was 
too hard to define what constitutes a "right” answer. This feedback drove rubric revisions and 
forced central office staff to work late into the night to revise the rubric to place it in more 
than 3,500 binders that would be distributed to teachers the next day. 

When MEs talk about their work, they emphasize their efforts to support teachers. They talk 
about being “excited when teachers are sponges” and how rewarding it is to have an experi- 
enced teacher tell them, “This is the first time I’ve ever had anyone who knows special educa- 
tion give me feedback” Through their work they are realizing that "lots of teachers haven’t 



from outside DCPS. 



MASTER EDUCATOR SCREENING PROCESS 
(# INVOLVED) 



Stage 1- Resume and essay (800+) 

Stage 2- Phone interview (520) 

Stage 3- A day Performance 

Assessment: teach sample lesson 
and debrief; observe and analyze 
video of teaching and provide feed 
back to teacher; present own student 
achievement data; analyze data; 
writing sample (260) 

Stage 4 - Final interview (70) 
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heard much good about their practice.”They talk about "being so impressed when a teacher 
can process a low score and engage in learning” and the challenge of working with a teacher 
who "lacks content knowledge,” or of observing in a classroom where “stuff is happening but 
there is no actual lesson.” MEs say they rarely observe classrooms that reflect the extremes— 
fabulous or horrible teaching. As a result, much of their work lies in helping teachers identify 
areas for growth. This takes many forms: helping a teacher who is perceived as “good" to 
think about the relationship between her practice and student results and any disconnect 
between the two; or helping a teacher who is great at guided reading get comfortable with 
her need to improve her math teaching skills. 



MEs’ professional development happens twice a month fortwo hours; rubric-norming, prob- 
lem solving, and getting feedback from one another are the topics on the agenda. Addition- 
ally, the MEs support one another between meetings. They describe sharing write-ups with 
one another for comment and e-mailing and talking at all hours as they puzzle through 
particularly tricky cases. Reflecting on their experience in first-year implementation of the 
framework and IMPACT, one of the MEs summed it up this way: “We were told it would be a 
bumpy ride and we would need to figure it out. We knew it and planned for it.” 

The bumps showed up in several places. In the first round of ME post-observation confer- 
ences in year one, MEs noticed differences in how teachers approached the conferences and 
the impact these differences had on the meetings. Some teachers assumed a passive role in 
the conference, waiting for the ME to tell them their score. ME’s interpreted this as a sign of 
teachers’ anxiety about the process and/or their lack of experience talking about instruction 
with a supervisor. Other teachers reflected on their lesson in advance of the conference and 
engaged the MEs as thought partners. In the second round of observations, the post-obser- 
vation conferences were more reflective and teachers were more engaged as they became 
familiar with the process and anxiety lessened. 



WHAT IS MOST 
STRIKING TO MASTER 
EDUCATORS IS THE 
POWER OFTHE 
RUBRIC TO CREATE A 
COMMON LANGUAGE 
ACROSS THE SYSTEM 
ABOUT INSTRUCTION. 



What is most striking to MEs is the power of the rubric to create a common language across 
the system about instruction. Teachers across the city are talking about checking for student 
understanding and targeting multiple learning styles. They are also sharing strategies within 
and across schools. MEs are also very aware of how this new system puts clear responsibil- 
ity on teachers for their performance. As one ME explained, “The onus is on teachers. We’re 
giving feedback. Now they need to do something with that information.” 



MEs play a delicate role, with one foot in the world of teachers and the other in the world of 
administrators. They are expected both to evaluate teachers and to support their develop- 
ment. In a school system like DCPS, where support and accountability have historically been 
seen as separate things provided by different people, MEs are pioneers in the effort to change 
that culture. As one ME explained, "Initially, we had to figure out how to go into classrooms 
and assure teachers that we’re trying to draw them in, not filter them out and what that 
means for professional development. This is not a ‘gotcha.’” 



The tension around the ME role is reflected in their relationships with instructional coaches 
(the people whose primary job responsibility is to support teacher development) and princi- 
pals. Because of their role in evaluating teachers, MEs are part of the administrators’ union. 
For this reason, the Washington Teachers’ Union worked to establish a firewall between MEs 
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and instructional coaches, discouraging communication between the two groups. So while 
MEs know a lot about the skills and talents of teachers and the areas where they would 
most benefit from support, they aren’t able to share this information with coaches, who 
are responsible for supporting teacher development. As a result, the support teachers are 
getting from coaches is less focused and effective in raising their level of performance than 
it could be. In the end, teachers pay the price because the formal observations continue and 
scores are calculated whether teachers are getting differentiated support based on evalua- 
tion data or not. And this dynamic reinforces a sense, on the part of many teachers, that the 
Teaching and Learning Framework is really just about accountability. 

The relationship between MEs and principals is similarly guarded. They observe and assess 
the same teachers, but there is little collaboration or coordination between MEs and prin- 
cipals, largely to ensure the objectivity and integrity of both people’s evaluation ratings. 
MEs do not see principals’ scores and they don’t discuss their scores until they have writ- 
ten their report and debriefed with the teacher. Occasionally, the principal requests support 
from MEs for specific teachers; this most commonly happens when the ME has expertise in 
a high-needs area, e.g. special education or high school science. MEs’ success relies in part on 
their ability to be and be perceived as neutral third parties in the evaluation of teachers. This 
requires carefully negotiating relationships with principals and instructional coaches. 



Calibration, Calibration, Calibration 

A PowerPoint slide is flashed up on the screen at the beginning of the DCPS Principals Acad- 
emy training session in March 2010. It lists every standard and sub-standard (13 in all) of 
the Teach element of the framework and compares the average scores of principals and MEs 
for each. These averages have been calculated based on the 11,008 assessments (99 percent 
completion rate) principals and MEs have done at this point in the first year of IMPACT imple- 
mentation. 

DCPS is collecting all of the classroom observation scores electronically and has the capac- 
ity to report scores by principals and MEs and to make very precise comparisons using the 
IMPACT information management infrastructure. These data allow the system to have 
evidence-based conversations about principals’ and MEs’ ratings and teacher practice, and to 
track and report data over time to see how the work is evolving. 

In reviewing the comparative data, the principals can see two striking findings. First is the 
variation in the scores given to different standards. For example, principals’ highest rated 
standard is Interacts Positively and Respectfully with Students (3.56 on a l-to-4 scale), and 
their lowest-rated standard, Probe for Higher-Level Understanding, is a full 0.8 lower (2.74). 
This discrepancy suggests that principals and MEs are making distinctions between perfor- 
mance in different standards and discerning trends in specific elements of instruction. These 
kinds of data are valuable in informing professional development priorities. Second, there is 
great similarity in the average scores of principals and MEs. On 10 of the 13 standards, the 
level of variation between principals’ and MEs’ scores is 0.2 or less, on a 1 to 4 scale. The varia- 
tion on five of the 13 standards is 0.1 or less. That suggests that the training on the rubric 
might be effective. 
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How the system presents the data and reacts to it is telling. There is a brief celebration and 
then principals are guided in homing in on T5c and T7, the two standards where there is the 
greatest variation in scores. The work of the day is trying to understand and address the root 
cause of this variation. Is there a lack of shared understanding about what each of the stan- 
dards means? Is there lack of clarity about what each looks like in practice when executed 
well? Principals spend the day reviewing the meaning of T5c and T7 and observing videos of 
teaching focused on those standards in an effort to increase the consistency of scoring. 

This emphasis on building a common understanding of what these standards look like in 
practice and calibrating scoring has permeated the first year of IMPACT implementation. 
Calibration is critical to ensuring the integrity of observation scores and the fairness of the 
teacher evaluation process. Much of the professional development tied to the framework 
and IMPACT in the first year has focused on observing videos (or live instruction) and calibrat- 
ing observations and scoring. This has been part of every Principals Academy session, every 
ME training, and much of the coach training. The teacher training has focused on teaching 
the strategies that embody the standards, using video footage of teaching wherever possible 
to bring the strategies to life. 

In the process of calibrating the scores of principals and MEs, a couple of interesting patterns 
have emerged. Most notably, the majority of scores for both principals and MEs are relatively 
high. On the l-to-4-point scale of the rubric, average scores hover just above 3.0. It is also 
noteworthy that principals’ scores on 10 of the 13 standards are higher than MEs’ scores. 
Tracking these trends and figuring out what they can teach the system about IMPACT, first- 
year implementation, and possible areas for refinement and professional development in 
Year Two will strengthen the performance management system. 



Calibration is also happening in schools. Because of IMPACT, principals and assistant prin- 
cipals are observing classroom instruction together and calibrating their ratings. For many 
administrative teams this is an entirely new practice that is encouraging a much clearer focus 
on instruction and greater collaboration among team members. As one principal described it, 
"We are getting better at it as a team and really building our leadership team.” 

Calibration and collaboration have also changed the work area superintendents do with the 
principals they supervise and evaluate. The framework and accompanying rubric are guid- 
ing area superintendents’ school walkthroughs with principals and focusing the conversa- 
tion on instruction. When asked whether classroom instruction has changed this year, one 
area superintendent described it this way: "I’m seeing more teaching and less unethical stuff. 
There are some nuances beyond the basics: seeing more checking for understanding, atten- 
tion to learning styles. By explicitly calling these things out in the framework, teachers are 
attending to them more.” 



CALIBRATION IS 
CRITICALTO ENSUR- 
ING THE INTEGRITY OF 
OBSERVATION SCORES 
ANDTHE FAIRNESS OF 
THE TEACHER EVALUA- 
TION PROCESS. 
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VOICES FROM THE FIELD 



Principals’ Perspectives 

Asking a broad sample of principals who represent different experience levels and schools in 
DCPS about TLF and IMPACT was a bit like opening the valve on a fire hydrant. As the people 
most responsible for implementing the initiative, and managing how the new expectations 
affect instruction and the climate and culture in their schools, they have strong opinions and 
are feeling a lot of pressure from the system and teachers alike. While there were points of 
agreement in the principal focus group it was also clear that their experience with TLF and 
IMPACT was directly related to the context of their school and their experience and expertise 
as a principal. These factors impacted how they led the implementation of the initiative and 
anticipated and managed teachers’ stress about the new work. 



How did TLF and IMPACT affect their work? One principal commented, "How I spend my time 
is very different. The jury is still out if that is for the better or worse. I'm doing lots more struc- 
tured evaluations and having great conferences with teachers multiple times a year. I’m less 
regularly in and out of classrooms, checking in, tracking instruction. I felt more involved last 
year. I’m not sure what of this is related to year one implementation.” A principal of a larger 
school that has three assistant principals described splitting up the teachers among the four 
administrators. The administrators observed classrooms together and calibrated their rating 
to ensure teachers would have a similar experience regardless of who observed them. She 
commented, "I can see us getting better as a team. We’re building a leadership team in a way 
we hadn’t ever done before.” 



Discussion of the rubric itself provoked lively discussion. A new principal leading a school 
where “teachers think everything is going well but student performance doesn’t match” 
found the framework provides a common language and a helpful starting place as he tried 
to set expectations. While one principal recognized that “some teachers see the value of TLF,” 
another principal complained, "For more sophisticated teachers this feels like 101. It’s frus- 
trating to high-performing teachers.” Concerns expressed about the rubric focused on both 
the content and how it is being rolled out and interpreted. There was a lot of concern about 
rigor. One principal described TLF as “not asking a whole lot. It lacks rigor.” Another said, "It’s 
fine to talk about pedagogy but we’re not talking about content." A third principal tied the 
previous two comments together, offering: “My biggest criticism is the lack of rigor which is 
tied to the lack of curriculum.” 



As the conversation shifted to the rollout of TLF, a principal expressed concern about focusing 
"on the little pieces.” She worried, “We’re losing sight of the holistic, big picture. There’s no 
conversation about rich pedagogy.” Principals talked about teachers’ being "really stressed 
out” and a number of principals agreed that the first year "should have been a pilot year. 
We shouldn’t be making dismissal decisions.” At the same time that principals expressed 
concerns about TLF and IMPACT, they also showed that they recognized their potential. As 
one principal observed, "Principals are asking about IMPACT scores when they think about 
teacher hiring.” 
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Teachers’ Perspectives 

In a focus group to discuss Year One implementation of TLF and IMPACT, a broad sample of 
DCPS teachers agreed that the framework provides clear expectations, which are helpful and 
did not previously exist. Ateacher explained, "The examples [in the rubric] of what I needed 
to do were helpful and I knew what was expected of me.” 

As the conversation shifted to the rollout and implementation of the system it was clearthat 
these teachers and the creators of TLF and IMPACT would surely agree that implementing 
IMPACT system wide was an undertaking that required enormous capacity which was hard 
to guarantee. Several teachers in the focus group wondered why the system hadn’t piloted 
TLF and IMPACT or introduced TLF one year and evaluation the next. Teachers described their 
initial introduction to TLF as less than a day of professional development with trainers who 
used PowerPoint slides and a script. This approach, which the system pursued to ensure a 
clear and consistent message, was perceived by teachers as canned and described as low 
level instruction - know and understand - on Bloom’s Taxonomy. Teachers described train- 
ers who were unable to answer probing questions, such as what was meant by the word 
“rigor.” Several teachers were frustrated by the lack of training materials. A third-year teach- 
er explained, "There were no videos of what it looked like to teach at a level 4. I need to 
see an example of someone excelling and achieving... not just [be told], if you do this it will 
happen.” 

The majority of teachers in the focus group described significant variations in their experi- 
ences with different evaluators. One explained, "After my first conversation with my Master 
Educator I felt it was going to be a worthwhile. She offered me some good resources. I thought 
I would get something out of it. My second Master Educator was kind of a robot, not gener- 
ous in offering assistance, [a] much tougher grader, colder, harder and more linear than the 
first.” Another teacher explained, "My conversation with my principal was supportive but 
not helpful. [She said,] ‘You did awesome’ with no suggestions for how to improve. My ME 
conversations were very meaningful. She helped me change many of the things that I did. I 
asked for this ME a second time because she had great things to say and was beneficial to 
me.” 

The subjectivity of scoring was a point of conversation among the teachers. Most of the teach- 
ers in the room either personally knew of a teacher who had received wide-ranging scores or 
had heard of such a teacher through the grapevine. One teacher spoke of the scores her own 
child’s teacher received. She characterized the teacher as phenomenal and said he got a 3.75 
score in SY TO, indicating that he was highly effective. In the fall of 2010, though, he was 
given a score of 1.55. The lower score was changed to a 2.1, which the teacher described as 
the evaluator’s response to seeing his “highly effective” plaque on his classroom wall — his 
reward for his previous score. While stories of these inconsistencies spread across the system, 
DCPS examined scores to identify the extent of this issue and found significant differences in 
scores in several dozen teachers (less than one percent of the teaching force). 

Teachers had great concerns about how IMPACT is affecting teaching. One teacher told the 
story of being padlocked out of the school auditorium and told that staging a production 
of Romeo and Juliet was not an "IMPACT-ready” lesson. She was advised by a school admin- 
istrator to send kids who present behavioral challenges to the library when she’s observed, 
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in order to help her with her score. She summarized her experience saying, "It’s disappoint- 
ing. I'm so unhappy about how it has affected my instruction. I feel a bit crushed under the 
weight of IMPACT.” A teacher who works in two-hour teaching blocks described her concern: 
“When they come in everything has to change for the 30 minutes they’re in there. You can 
do X, Y, and Z perfectly but if you run out of time and don’t get back to Ayou're penalized.” A 
third teacher explained that her school for the first time did not make Adequate Yearly Prog- 
ress (AYP) under the federal No Child Left Behind Act, and attributed that result to IMPACT 
and how teachers changed their instruction in response to it. As she explained, "Teachers 
stripped lots out of their instruction. They dried it up and bored the heck out of kids.” 

The issue of equity was also raised in the conversation. Teachers posited that it is much harder 
to get good scores in a school in trouble. A teacher explained, “There is a difference between 
schools. Some schools have huge behavior issues. A teacher with a 3.7 score in a school with 
no issues could come into my school and teach the same lesson because, supposedly, this 
person is a highly effective teacher. But this is impossible. The context of the school matters. 
None of them could come into my classroom and do what I did and even get close to the 
score that I did. This is a flaw in the system design. It doesn’t have to do with the rubric. No 
one is going to admit this because it’s a huge political problem.” 
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YEAR-ONE IMPLEMENTATION RESULTS 

At the end of the first year of implementing TLF and IMPACT, DCPS officials examined three 
critical sets of data to determine the progress made in Year One implementation and the 
priorities moving forward. They were: 1) the level of consistency in evaluators’ scoring class- 
room observations using the TLF rubric; 2) the correlation between TLF scores and individual 
value-added scores; and 3) the overall IMPACT ratings for all the teachers in the system. 



Scoring and Correlation 

Throughout the year, the IMPACT information management system synthesized the evalua- 
tors’ rubric scoring. At the end of the year, these data were able to show variations in teach- 
ers’ performance across standards, as well as variations between principals’ and MEs’ ratings. 
These two analyses can be very helpful in informing teacher professional development, 
training of principals and MEs on specific standards, and rubric refinement. The table below 6 
shows what the averages looked like in June 2010. It represents more than 17,000 observa- 
tions and a full year of implementing TLF and IMPACT. 



STANDARD 


PRINCIPALS’ 

AVERAGE 

SCORE 


MASTER 

EDUCATORS’ 

AVERAGE 

SCORE 


Tl: 


Focus Students on Lesson Objectives 


2.97 


2.74 


T2: 


Deliver Content Clearly 


3.02 


2.71 


T3: 


Engage all Students in Learning 


3.4 


3.38 


T4: 


Target Multiple Learning Styles 


3.07 


2.98 


T5a: 


Check for and Respond to Student Understanding During the Lesson 


3.06 


2.91 


T5b: 


Respond to Student Misunderstanding 


3.13 


2.95 


T5c: 


Probe for Higher-Level Understanding 


2.81 


2.36 


T6: 


Maximize Instructional Time 


3.1 


2.92 


T7: 


Invest Students in Learning 


3.08 


2.63 


T8: 


Interact Positively and Respectfully with Students 


3.51 


3.52 


T9a: 


Student Behavior 


3.29 


3.28 


T9b: 


Reinforce Positive Behavior 


2.97 


2.82 


T9C: 


Address Inappropriate, Off-Task, or Challenging Behavior 


3.37 


3.4 


Total 


3.15 


2.97 







While a comparison of average scores is helpful in showing trends, the use of aggregate aver- 
ages makes it impossible to see the level of variability at the level of an individual teacher 
who was observed by a principal and two different MEs. To examine this, DCPS used a corre- 
lation analysis that measured the extent to which principals’ scores related to MEs’ scores. 
A correlation of 0 suggests there is no relationship between the two sets of scores while a 
correlation of 1 means there is a perfect correlation and the two sets of scores move up and 
down in unison. The correlation between administrators’ and MEs' scores at the end of year 1 
was 0.57, which is thought to be a moderately strong correlation in social sciences. 7 
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6 This table reflects version 1.0 
of the framework which was 
used in SY 2009-10. 

7 0.50 is generally agreed upon 
as the floor for a "moderately 
strong" correlation. 
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The story became more complicated when DCPS 
analyzed the relationship between TLF rubric scores 
and individual teacher value-added scores based on 
the DC-CAS. At this early stage in the use of value- 
added analysis nationally the hope is that there is 
a strong correlation between a teachers’ score on an 
instructional rubric and his or her value-added score. 
This would validate the instructional rubric by showing 
that doing well in instruction produces better student 
outcomes. DCPS analysis at the end of the first year 
of IMPACT suggests that there is a modest correlation 
between the two ratings (0.34). 

Master educators’ scores and individual value-added scores show the highest correlation, 
0.08 higher than the correlation for administrators. There are several possible reasons for 
this: 1) Principals include in their ratings factors that are not part of the rubric but that they 
are aware of because they know the teachers; 2) MEs are particularly expert in the observa- 
tion and analysis of instruction and therefore hold teachers to a higher standard than prin- 
cipals. 

DCPS’s correlations are similar to those of other districts that are using both an instructional 
rubric and value-added data. A moderate correlation suggests that while there is a correla- 
tion between the assessment of instruction and student learning as measured by standard- 
ized tests (for the most part), it is not strong. At this early stage of using value-added data 
this is an issue that needs to be further analyzed. 



Correlation Between Individual 
Value Added (IVA) and TLF 8 




Teacher Ratings 

The third set of data relates to DCPS teachers’ IMPACT scores, which, as noted above, reflect 
the synthesis of the observation scores, value-added data where available, and the other 
measures: commitment to school community, school value-added and core professionalism. 
The overall IMPACT rating scale runs from 100 to 400 and rates teachers at four levels: inef- 
fective, minimally effective, effective, and highly effective. In setting the cut scores for these 
levels, DCPS made Effective the widest band and Highly Effective the smallest, sending the 
message that a rating of Highly Effective was a special mark of distinction. The first chart 
illustrates both the IMPACT scale and the percent of teachers who scored at each of the four 
rating levels at the end of the 2009-10 school year. The second chart reflects the performance 
categories for teachers in 2009, before IMPACT was implemented, and the distribution of 
teacher ratings in that school year. 



8 IMPACT Presentation 
PowerPoint, DCPS-TNTP 
Conference, October 5, 2010 



The 2008-09 data make clear that DCPS had had the same problem with teacher evaluation 
that The New Teacher Project identified in school systems across the country in its report, The 
Widget Effect: a tiny proportion of teachers were identified as unsatisfactory. Nearly half of 
the DCPS teacher workforce was identified as performing at the highest level. IMPACT was 
developed to address this issue by providing a more robust and higher standard for teacher 
performance. At the end of the first year of implementation of IMPACT, the trends shifted, 
with the proportion of teachers rated Effective or better decreasing from 95 percent to 81 
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Comparison of Teacher Ratings Under 1st Year of IMPACT and Previous Year Results 




percent and the proportion of teachers identified as less than effective increasing from 5 
percent to 19 percent. 

Digging into the data, it is interesting to note that there is little variation in the distribution 
of Ineffective and Highly Effective teachers based on tenure in the system, with the excep- 
tion of a lower percent ofteachers in theirfirst sixyears ofteaching performing at the Highly 
Effective level. 

The ratings have consequences attached to them. Teach- 
ers identified as Ineffective can be dismissed within a year. 

In June 2010, DCPS dismissed 125 teachers who were rated 
Ineffective in the fall of 2009. Teachers rated Minimally Effec- 
tive have two years to improve their rating. The 700 teachers 
rated in this category in 2009 have until June 2011 to demon- 
strate effectiveness; those who do not will then be eligible 
for dismissal. 



For Highly Effective teachers, DCPS wanted to recognize and 
reward their talent. The mechanism for recognition that the 
system chose was compensation. The IMPACTp/us compen- 
sation system was designed to provide significant financial 
rewards to the system’s Highly Effective teachers by aligning 
compensation to performance. 



Percent of Ineffective and Highly Effective 
Teachers Based on Years of Experience 



TEACHER TENURE 
IN THE DCPS 


INEFFECTIVE 


HIGHLY 

EFFECTIVE 


<6 years 


2.7% 


12% 


6-10 years 


0.6% 


22% 


10-20 years 


2.0% 


19% 


20-30 years 


2.3% 


16% 


>30 years 


3.4% 


15% 



IMPACTp/uJ 

DCPS’s salary structure, prior to the introduction of IMPACT, was organized on the same 
principles as most school systems’ pay schedules. Teachers earned salary increases based on 
their years of service and their accrual of graduate credits, two factors that have been shown 
to have little effect on teacher performance or student achievement. In the spring of 2010, 
DCPS and the Washington Teachers’ Union, after two and a half years of contentious nego- 
tiations, agreed to a contract that introduced a compensation system aligned to IMPACT. All 
new teachers would be enrolled in the new system; current teachers could choose whether 
or not to enroll in it or maintain their existing pay structure. 
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Through IMPACT plus, Highly Effective teachers can earn an annual bonus of up to $25,000 
and accelerate the growth of their base salary. All Highly Effective teachers move up five 
steps on the traditional compensation ladder for every two years they are evaluated as High- 
ly Effective. Their bonuses are calculated based on effectiveness, teaching the most economi- 
cally disadvantaged students (who tend nationally to be taught by the least-experienced and 
least-effective teachers), and teaching in high-stakes testing grades and high needs subject 
and program areas. These criteria recognize teachers as the most important school-based 
factor in students’ learning and incentivize Highly Effective teachers to work where their 
talents are most needed: the highest-needs schools, the highest-needs subjects, and grades 
with value-added data. DCPS’s pay-for-performance system focuses on leveraging teaching 
talent to serve students rather than the traditional pay system’s focus on teachers’ tenure in 
the system and their accrual of graduate credits. 



The chart below shows how bonuses are calculated for Highly Effective teachers who opt 
into IMPACTp/us. 





SCHOOL’S FREE 




ADD-ON IF 


ADD-ON IF YOU 


TOTAL POSSIBLE 


1 


IMPACT SCORE 


AND REDUCED 


BONUS 


IN IMPACT 


TEACH A “HIGH 


ANNUAL 






PRICE LUNCH RATE 




CROUP l 10 


NEEDS” SUBJECT 


BONUS 





Highly Effective 


60% or higher 


$10,000 


$10,000 


$5,000 


$25,000 




59% or lower 


$5,000 


$5,000 


$2,500 


$12,500 



At the same time that the new teachers’ contract provides Highly Effective teachers the 
opportunity to dramatically increase their earnings, it also includes a "mutual consent” provi- 
sion that eliminates job security for teachers who lose their positions. Underthe agreement, 
teachers who are laid off who have a rating of Effective or Highly Effective are given a year at 
full pay to find a job, the option of a $25,000 buyout, or early retirement with full benefits if 
they have 20 or more years of service. However, if highly rated teachers choose to participate 
in IMPACTp/us, they would not have access to these three options. In effect, teachers trade 
that safety net for the opportunity for significantly increased earnings. 

At the end of the first year of implementation of IMPACT, 501 classroom teachers in DCPS 
were eligible to participate in IMPACT plus based on their Highly Effective rating. Chancel- 
lor Rhee had raised $31.5 million in private foundation money to pay for IMPACTp/us and 
anticipated paying out $6 million in bonuses in the first year. Sixty-three percent (318) of the 
eligible teachers chose to participate in the new performance-pay system. For the teachers 
who opted in, the average bonus was $9,190, and the system paid a total of $2.9 million in 
bonuses to classroom teachers in the fall of 2010. Two DCPS teachers were eligible for the full 
$25,000 bonus; both chose to take it. 

A third of the eligible teachers chose to stay with the traditional, incremental base-build- 
ing compensation system. The participation rate of Highly Effective teachers in IMPACTp/us 
reflected the enormity of the shift in thinking the bonus-pay system required teachers to 
make as they traded job security for significant pay increases. It likely also reflected weak 
trust in the system by some teachers, the result of acrimonious contract negotiations, and 
the perception of IMPACT as a tool to support teacher dismissal. 



DCPS TLF AND IMPACT: EXPECTATIONS, ACCOUNTABILITY, AND SUPPORT 



9 IMPACT Group 1 is the group 
of teachers for whom there is 
DC-CAS value-added data. 
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LEARNING AND REFINEMENT 

The accomplishments achieved in the first year of TLF and IMPACT are impressive in both 
their breadth and depth. DCPS introduced district-wide standards for teaching and an aligned 
rubric. Every teacher in the system was observed five times using the new rubric. The role of 
Master Educator was implemented, ensuring a second observer for every teacher and intro- 
ducing evaluators who had most recently been classroom teachers. DCPS initiated a new 
teacher evaluation system that includes value-added measures and a new compensation 
system that aligns pay to performance. 

Given all the system had taken on in school year 2009-10, district leaders were reluctant to 
start any new initiatives the following year. As Michael Moody described the district’s Year 
Two message about TLF and IMPACT, the goal was to "get better at what we started.” At the 
end of that first year, there was plenty of experience and data to draw on to define the priori- 
ties for refining and deepening implementation in year two. 



Framework and Rubric 

DCPS decided to spend the second year of implementation continuing the Year One focus 
on the Teach element of the framework, and putting off a deep exploration of "Plan” and 
“Increase Effectiveness.” There was work to be done in refining the framework and rubric 
and in deepening everyone’s understanding of it and ability to teach to it. Year One data 
from observations and from more than 50 focus groups with teachers about the frame- 
work and rubric pointed out areas for improvement in both the content of the rubric and its 
language. 

In terms of content, teachers consistently scored the lowest on the standards related to high 
expectations, rigor, and probing for higher-level understanding (T5c and T7). Given how criti- 
cal these skills are to supporting students to do challenging and rigorous work, the district 
had to amplify these skills in the framework revisions and professional development. 

Feedback from teachers and their evaluators informed revisions to the format and language 
of the rubric. Clear themes emerged from the feedback. First, the standards and rubric need- 
ed to be refined and streamlined to be less cumbersome to use and understand. There were 
points of repetition that needed to be eliminated and the rubric needed to be made more 
flexible. For example, the comments suggested that the statement "Teacher attempts to 
target 3 or more learning styles and effectively targets 3” should be changed to something 
less rigid, such as “Teacher provides students multiple ways to engage with content.” Second, 
rather than rating each element of a standard discretely and then calculating the rating for 
that standard, the feedback suggested that the standards needed to be framed more holisti- 
cally, with evaluators giving an overall score for each standard. Third, the standards language 
and their description in the rubric needed to be revised to more precisely focus on what eval- 
uators had learned in Year One were the most important things to focus on relative to each 
standard. These changes are evident in the tuning of the standards themselves as well as in 
the details of the rubric. As an example, T3 was revised from "Engage all students in learning” 
to "Engage students at all learning levels in rigorous work.”The emphasis shifted from simply 
engaging all students to focusing on differentiation and rigor. 
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TLF 2.0 was released in the summer of 2010. 

The grid below illustrates the evolution of the standards: 



2009-10 TEACH DOMAIN STANDARDS 


2010-11 TEACH DOMAIN STANDARDS 


Tl: 


Focus students on lesson objectives 


Tl: 


Lead well-organized, objective-driven lessons 


T2: 


Deliver content clearly 


T2: 


Explain content clearly 


T3: 


Engage all students in learning 


T3: 


Engage students at all learning levels in rigorous work 


T4: 


Target multiple learning styles 


T4: 


Provide student multiple ways to engage with content 


T5A: 


Check for and respond to student 
understanding duringthe lesson 


T5 : 


Check for student understanding 


T5B: 


Respond to student misunderstanding 


T6: 


Respond to student misunderstanding 


T5C: 


Probe for higher-level understanding 


T7 : 


Develop higher-level understanding through effective 
questioning 


T6: 


Maximize instructional time 


T8: 


Maximize instructional time 


T7 : 


Invest students in learning 


T9: 


Build a supportive, learning-focused classroom community 


T8: 


Interact positively and respectfully 
with students 






T9A: 


Student behavior 






T9B: 


Reinforce positive behavior 






T9C: 


Address inappropriate, off-task, or 
challenging behavior 







Expectations, Support and Accountability 

The second clear area of focus for Year Two related to providing teachers and principals more 
support organized on the standards. This priority implicated instructional coaches, princi- 
pals, and assistant superintendents. Coaches who had limited training on the framework and 
rubric in SY 2009-10 had often worked with teachers on a parallel track, untethered to the 
framework. Several changes were proposed for Year Two to bring coaches’ support of teachers 
into closer alignment with the standards. In SY 2010-11 a more rigorous central screening of 
coaches that principals could choose from was implemented, and coaches received substan- 
tive training on the framework. They were expected to integrate the framework into their 
coaching and 30 percent of their evaluation was tied to the growth of average TLF scores of 
teachers in the schools they serve and the school value-added student achievement data. 

As district officials thought about the role of principals, they realized they simultaneously 
needed clearer expectations and more support. Clearer expectations came in the form of 
a new principal evaluation developed in SY 2010-11. In this new evaluation, 50 percent of 
principals’ performance rating is based on student achievement gains (proficiency, growth, 
and school-level measures). Five percent is tied to the school’s retention of highly effective 
teachers. To support principals to meet these expectations, DCPS doubled the number of 
instructional superintendents who supervise and evaluate principals. In SY 2010-11, each of 
the 12 instructional superintendents work with 10 to 12 schools. They do instructional walks 



26 



DCPS TLF AND IMPACT: EXPECTATIONS, ACCOUNTABILITY, AND SUPPORT 



THE ASPEN ) INSTITUTE 



EDUCATION & SOCIETY PROGRAM 



with every principal every other week, work with them to closely analyze student learning 
data, and consider implications for instruction, and provide principals monthly written feed- 
back. The number of MEs was also increased from 32 to 42 to meet the demand of observa- 
tions and six senior MEs were identified and assigned to lead content-based teams of MEs to 
provide them more support and assistance. 

To further build the capacity in schools, assistant principals (APs) were integrated more fully 
into the training and development related to TLF and IMPACT. In SY 2009-10 several sessions 
were held for assistant principals on the framework and IMPACT; in SY 2010-11, every month- 
ly principal leadership academy session is followed by a session for APs that covers the same 
content. 



There are clear SY 2010-11 professional development priorities to ensure that teachers, prin- 
cipals, and coaches are well versed in the framework and to provide professional develop- 
ment on the standards in which teacher performance is weakest. At the same time, there is 
an effort underway to build on this professional development framework. TLF articulates the 
elements of effective instruction generically, which creates system-wide understanding and 
expectations regarding instruction that will likely raise the floor for teachers’ instructional 
practices. Yet to sustain improvements in teacher practice and student achievement, teach- 
ers need professional development to integrate the ideas embedded in the framework into 
teaching in the content areas. For example, what are the strategies for checking for student 
understanding and respondingto misunderstanding in a guided reading lesson? How doyou 
provide students in an Algebra 1 class multiple ways to engage in the content? DCPS began 
this work this fall by having MEs facilitate six week, 90-minute sessions of groups of about 20 
teachers organized by content area (e.g. elementary math, high school English) that focused 
on T1 and how it is expressed in their content area. Between sessions, participants tried out 
things they discussed in session and brought their experiences back to the group to debrief. 
This is a first step. 



TO SUSTAIN IMPROVE- 
MENT, TEACHERS 
NEED PROFESSIONAL 
DEVELOPMENT TO 
INTEGRATE THE IDEAS 
EMBEDDED INTHE 
FRAMEWORK INTO 
TEACHING INTHE 
CONTENT AREAS. 



System Infrastructure and Organization 

Given that 86 percent of DCPS teachers do not teach content or grades in which he DC-CAS 
is administered, the issue of how student learning is assessed is both messy and critical. 
Because DCPS leaders were unsure how to proceed with these teachers, they made achieve- 
ment based on non-DC-CAS scores a small part of the evaluation (10 percent of the rating, 
compared with 50 percent for teachers with DC-CAS value-added scores). In SY 2009-10, 
teachers were expected to bring data to principals three times a year that showed evidence 
of student learning. Principals were expected to determine the level of learning represented 
in the students' work and then give the teacher a score of 1 to 4 for this element of IMPACT. 
The district provided a rubric to help guide teachers in selecting the assessment data they 
brought and to support principals in assessing the data. There were huge variations in what 
teachers submitted and how principals assessed it. This raised fundamental questions about 
the validity of these measures, as well as knotty questions like: what constitutes 1.5 years of 
learning on a seventh grade science test and how doyou measure it? The system didn’t have 
clear answers to these questions and principals didn’t know how to discern which assess- 
ments would be most useful for assessing student learning. 
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To begin to address this issue, in SY 2010-11, the district established guidelines that required 
every teacher to propose at the beginning of the year what assessments they would use to 
measure student learning and what measures of progress they will hold themselves account- 
able for. The principal has to approve the proposal. For example, a teacher might administer 
a final exam and set an expectation that 80 percent of the students will score at 80 or better 
on the exam. Until the district develops district-wide assessments, the system will have to 
continue to rely on teacher-developed and-administered assessments. In the meantime, 
DCPS put out guidelines about what could be used as assessments by grade and content area 
to create more consistency. (These guidelines make clear that formative assessments, which 
are designed to guide instruction, will not be used for accountability.) Over the long run, 
district officials expect to develop common assessments for subjects and grades not covered 
by DC-CAS. Senior leaders suggest these will likely be put in place in the next several years. 

Implementation of the framework has shone a spotlight on an even bigger piece of teaching 
and learning infrastructure the system does not have in place: a common DCPS curriculum, 
K-12, to guide instruction. Instructional materials, which are not consistent across schools 
in the system and which invariably differ in quality and alignment to DC standards, often 
serve as the default curriculum. As the system has raised the expectations for teachers it has 
become clear that it needs to similarly increase the level of curricular support it is providing 
to guide teachers’ instruction. DCPS has adopted the Common Core State Standards which 
sets the direction for this curriculum work. 

A DCPS central office staff person perfectly captured the dilemma the system finds itself in 
regarding the teaching and learning infrastructure as it implements IMPACT commenting, 
“It’s easy to create requirements through evaluation. It’s much harder to create things to 
meet these requirements.” Developing assessments that can be used to measure student 
learning in classes that do not have DC-CAS data is one example of this dilemma. 

Evaluation is the wedge DCPS chose to provoke systemic reform. Now the system faces the 
challenge of building the infrastructure required both to fully implement the evaluation 
system and to provide teachers the full complement of curricular, instructional, and assess- 
ment supports they need to do their jobs well. This is the work in which the teaching and 
learning team is now engaged. 
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CONCLUSION 

DCPS’s development of its Teaching and Learning Framework and its use as a centerpiece 
of the larger IMPACT teacher performance management system is being watched closely. 
School systems across the country that need to tackle the same issues of defining instruction- 
al excellence and aligning support and accountability to it are looking to DCPS for guidance. 
Organizations that support public education and that are particularly focused on teacher 
effectiveness are watching to see if DCPS has created a viable solution to a heretofore intrac- 
table problem. And policy makers are tracking DCPS’s progress, considering the implications 
for local, state, and federal policy. 

Given all this attention, it is worth calling out some of the critical learnings from the DCPS 
experience that are generalizable and can inform others as they take on this work. These 
lessons include: 

1 Tap the potential of teaching standards to create common expectations and language 
about instruction and to focus everyone in the system on thinking about how teachers 
teach and why. 

Quite often, school systems have no common, system-wide understanding of what 
good teaching is. It is a black box that can’t be explained with any consistency across the 
system and evaluators talk about "knowing good instruction when they see it.” Without 
defining the standards of effective teaching, everyone in the system is at a disadvantage 
in their efforts to improve instruction and student learning. Teachers don’t know what 
they are working towards. Professional development providers choose training priorities 
ungrounded by a common vision for effective instruction. Principals can’t talk specifically 
and concretely about teachers’ strengths and weaknesses in the context of an overarch- 
ing set of performance expectations. With the introduction of the TLF, DCPS provided a 
vision of effective teaching around which everyone in the system could organize their 
work. After just a year of implementation, teachers, principals, master educators, and 
central office staff are talking in the language of the standards in ways that suggest a 
growing shared understanding of both the standards and how teaching practices in the 
system measure up against them. 



2 Anticipate that the hardest and most important part of a teacher performance manage- 
ment system is helping teachers improve their practice. 

Jason Kamras described the initial goal of IMPACT as identifying teachers along a perfor- 
mance spectrum and differentiating their treatment. The treatment of teachers at the 
high and low ends of the spectrum is the easiest to see initially as the system built a new 
compensation system to reward high performers and pursue dismissal of low perform- 
ers. 

With 82 percent of DC’s teacher workforce evaluated in the other two performance cate- 
gories-Minimally Effective or Effective-the questions the system now needsto answer 
are: How can it support these teachers to improve? What are the curricular, assessment, 
and professional development support teachers need to improve? How can teachers’ 
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work in schools be organized to support their continuous learning? What is the role of 
principals, coaches, and other personnel to support teacher learning and development? 

For school systems whose evaluation systems have been either nonexistent or funda- 
mentally broken for decades, developing a robust evaluation system is a tremendous 
accomplishment. Yet it is the first step in creating a robust performance management 
system. The next step is to imagine what a system looks like that is focused on high 
expectations for teacher performance and supporting teachers to continually improve 
their practice and be excellent. Creating a support system in conjunction with new 
teaching standards and evaluation systems has the potential to catapult both teacher 
and student performance. It requires vision and a willingness to put everything on the 
table. 

3 Consider the trade-offs associated with introducing teaching standards and an aligned 
evaluation system simultaneously as compared to sequentially. 

DCPS linked TLF and evaluation to get teachers’ and principals’ attention. From the initial 
introduction of the standards, teachers knew the stakes attached to them and paid atten- 
tion. Teacher conversation tended to focus on IMPACT and accountability rather than TLF 
as they fixated on the evaluation process. DCPS’s approach highlighted accountability 
and allowed the system to both celebrate excellent teachers and dismiss poor perform- 
ers at the end of the first year of implementation. 

The introduction of the framework has led to more focused and explicit conversations 
about instruction among teachers, principals, and everyone whose job it is to support 
them in implementing the framework and IMPACT. There is less evidence of deep and 
nuanced conversations about the standards, the interplay between them, how teachers 
can use them to improve their practice, and how principals can support that work. This 
may be a function of the fact that the system is in the early stage of implementation and 
is working to deepen what, for many, may be a fairly superficial awareness level of the 
standards. It may also be the result of people feeling such pressure that they just want 
to know what to do to get a rating of 3 or 4 in a particular standard. 

Decisions about the pace and sequence of implementation are deeply contextual and 
there is no empirical "right way.” The challenge is to simultaneously create a sense of 
urgency and the conditions needed to engage teachers and principals in the work of 
deeply and consistently examining and refining teaching. Disequilibrium is important 
to drive changes in behavior and meaningful learning. The trick is to create just enough 
disequilibrium to drive productive improvement without creating so much that the 
intended change is subverted. 

4 Understand that building a robust teacher performance management system is going to 
require the continuous development of organization capacity. 

Effectively implementing teaching standards requires training teachers and evaluators 
on the standards, building the infrastructure to capture the evaluation data, develop- 
ing accountability mechanisms, and creating checks and balances to ensure consistency 
in application. DCPS’s work illustrates that these needs can be anticipated, the system 
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can build the needed capacity to do this work, and doing these things is an enormous 
undertaking. 

DCPS’s experience also suggests that as systems pursue this work they will unearth 
significant, systemic issues of misalignment, insufficient capacity, and quite simply 
a lack of knowledge about how to do new work. In DCPS, the introduction of TLF and 
IMPACT made clear the need for better-aligned support for teachers and principals, the 
need for a system-wide curriculum and a system of interim assessments. The curriculum 
and assessment issues are complicated and labor-intensive, and get at the system’s core 
teaching and learning work. TLF and IMPACT highlighted these problems and forced the 
system to take notice. While it is not possible to fully anticipate all the implications of 
new teaching standards and an aligned evaluation system, it is wise to expect that there 
will be many, to try to anticipate them, and wherever possible to begin to organize to 
address them. 

5 Think strategically about organizing the system to drive the design and implementation 
of teaching standards, the associated evaluation system, and the related supports. 

DCPS’s new Office of Human Capital, which reports directly to the chancellor, is respon- 
sible for implementing IMPACT and IMPACTp/us. It was a strategic decision to create a 
division free of organizational baggage to drive innovation. The Office of Human Capital 
is described as a "dynamic, fast-moving department that is constantly evolving its work, 
making decisions at midnight that are put in place the next day.”The department faced 
the simultaneous challenge and opportunity of building and implementing IMPACT and 
IMPACT plus from whole cloth in a very short period of time. It was surely daunting and 
they accomplished a lot. 

The Department of Teaching and Learning, which created the framework on which 
IMPACT and IMPACTp/us were built, sits deep in the organizational bureaucracy and is 
encumbered by well-established systems, structures, and past practices. In addition to 
the framework, the department is also responsible for the district’s curriculum, instruc- 
tional and assessment infrastructure, and the supervision of principals. The pace of the 
Department of Teaching and Learning was described as “very different” from the pace of 
Human Capital. 

With the teaching standards set, Teaching and Learning faces the challenge of shedding 
old ways of doing work to address enormous pedagogical, content knowledge, coaching, 
and professional development needs surfaced by the introduction of the framework and 
IMPACT. The need to revamp an existing teaching and learning infrastructure to support 
teacher development in alignment to teaching standards is a dilemma most districts 
that embark on this work will face. 

DCPS’s experience raises important questions about how school districts organize 
to both design and implement new performance management systems: Does a new 
department, unencumbered by history and the bureaucracy, need to be created to 
drive this effort? If so, what is the scope of its authority and how can it be leveraged to 
support deep, systemic change required in other departments? How many departments 
are involved in the work? What are the levels of capacity, flexibility and autonomy of 
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each? What is the level of collaboration between the departments required for success? 
How are rewards and incentives for the departments organized to discourage depart- 
ments working as a series of silos and to support the performance management work? 
What are going to be the hardest parts of designing and implementing a performance 
management system and how can we organize in anticipation of them? These questions 
help the system anticipate the centrifugal force of a bureaucracy and organize the work 
to counteract it; this is crucial to long-term, systemic change. 



The work DCPS has undertaken in introducing the TLF and IMPACT is broad in its reach. The 
results realized at the end of Year One are promising. They speak to the potential of this 
initiative to fundamentally alter what it means to be an effective teacher in DCPS and to 
raise the quality of instruction and student achievement. There is much to be learned from 
DCPS’s work and its role as a frontrunner in redefining teaching standards and evaluation, 
long the Achilles’ heel of public education. The highest compliment school systems embark- 
ing on this work can pay DCPS is to learn from its efforts and then refine and increase the 
sophistication of this nascent work. 
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APPENDIX A: 

DCPS Teaching and Learning Framework 2.0 



u st.be guided by robust data. . 







INSTRUCTION 

1. Develop annual student 
achievement goals 

2. Create standards-based unit plans 
and assessments 

3. Create objective-driven lesson plans 

LEARNING ENVIRONMENT 

4. Adopt a classroom behavior 
management system 

5. Develop classroom procedures 
and routines 

6. Organize classroom space 
and materials 




1. Lead well-organized, objective- 
driven lessons 

2. Explain content clearly 

3. Engage students at all learning levels 
in rigorous work 

4. Provide students multiple ways to engage 
with content 

5. Check for student understanding 

6. Respond to student misunderstandings 

7. Develop higher-level understanding through 
effective questioning 

8. Maximize instructional time 

9. Build a supportive, learning-focused classroom 
communtiy 




INCREASE 

EFFECTIVENESS 

1. Assess student progress 

2. Track student progress data 

3. Improve practice and re-teach in response to data 



^7 dsop ojyfypqisuodssj pue 
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APPENDIX B: TLF RUBRIC (EXCERPTS) 

Teaching and Learning Framework (TLF) Rubric: Plan 

Note: In 2009-2010, only the TEACH domain of the Teaching and Learning Framework will be part of the teacher assessment process. 



LEVEL 4 (HIGHEST) LEVEL 3 



I^TLFPI: 


DEVELOP ANNUAL STUDENT ACHIEVEMENT GOALS 




TLF 

Plfl 


Teacher develops an ambitious and measurable annual 
student achievement goal for her/his class that is aligned 
to the DCPS content standards. 


Teacher develops a measurable annual 
student achievement goal for her/his class 
that is aligned to the DCPS content standards. 


TLF 

P1B 


All or nearly all students can communicate (in a 
developmental^ appropriate manner) the goal and how it 
will be assessed. 


Most students can communicate (in a 
developmentally appropriate manner) the goal 
and how it will be assessed. 


TLF P2: CREATE STANDARDS-BASED UNIT PLANS AND ASSESSMENTS 


TLF 

P2A 


Based on the annual student achievement goal, the 
teacher plans units by: 1) identifying the DCPS content 
standards that her/his students will master in each unit; 
2) articulating well-designed essential questions for each 
unit; 3) creating well-designed assessments before each 
unit begins ("beginning with the end in mind”); and 4) 
allocating an instructional^ appropriate amount of time 
for each unit. 


Based on the annual student achievement goal, 
the teacher plans units by: 1) identifying the 
DCPS content standards that her/his students 
will master in each unit; 2) articulating well- 
designed essential questions for each unit; and 
3) creating well-designed assessments before 
each unit begins ("beginning with the end in 
mind”). 


TLF 

P2B 


For any given unit, all or nearly all students can 
communicate (in a developmentally appropriate manner) 
the essential question(s) of the unit. 


For any given unit, most students can 
communicate (in a developmentally 
appropriate manner) the essential question(s) 
of the unit. 


TLF P3: CREATE OBJECTIVE-DRIVEN LESSON PLANS 


TLF 

P3 


Based on the unit plan, the teacher plans daily lessons by: 

1) identifying lesson objectives that are aligned to the 
DCPS content standards and connected to prior learning; 

2) matching instructional strategies to the lesson 
objectives; and 3) designing daily assessments that 
measure progress towards mastery. 


Based on the unit plan, the teacher plans daily 
lessons by: 1) identifying lesson objectives 
that are aligned to the DCPS content standards 
and connected to prior learning; and 2) 
matching instructional strategies to the lesson 
objectives. 



Each line of the rubric is assessed independently 
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LEVEL 2 LEVEL 1 (LOWEST) 



A 







Teacher develops a measurable annual student 
achievement goal for her/his class. 


Teacher develops a general annual student achievement goal 
for her/his class OR does not develop a goal at all. 


Half of the students can communicate (in a 
developmental^ appropriate manner) the goal and 
how it will be assessed. 


Less than half of the students can communicate (in a 
developmental^ appropriate manner) the goal and how it will 
be assessed. 






Based on the annual student achievement goal, the 
teacher plans units by: 1) identifying the DCPS 
content standards that her/his students will master in 
each unit; and 2) articulating well-designed essential 
questions for each unit. 


Teacher does not plan units by identifying the DCPS content 
standards that her/his students will master in each unit OR 
does not articulate well-designed essential questions for each 
unit. 


For any given unit, half of the students can 
communicate (in a developmental^ appropriate 
manner) the essential question(s) of the unit. 


For any given unit, less than half of the students can 
communicate (in a developmental^ appropriate manner) the 
essential question(s) of the unit. 






Based on the long-term plan, the teacher plans daily 
lessons by identifying lesson objectives that are 
aligned to the DCPS content standards. 


Teacher has little or no evidence of daily lesson planning 
based on the DCPS content standards. 




J 
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Teaching and Learning Framework (TLF) Rubric: Increase Effectiveness 

Note: In 2009-2010, only the TEACH domain of the Teaching and Learning Framework will be part of the teacher assessment process. 



LEVEL 4 (HIGHEST) LEVEL 3 



f TLFIE1 


: ASSESS STUDENT PROGRESS 




TLF 

IE1 


Teacher: 1) routinely uses assessments to measure student 
mastery of content standards; 2) provides students with 
multiple ways of demonstrating mastery (for example, 
selected response, constructed response, performance task, 
and personal communication); and 3) provides students 
with multiple opportunities during the unit to demonstrate 
mastery. 


Teacher: 1) routinely uses assessments to 
measure student mastery of content standards; 
and 2) provides students with multiple ways of 
demonstrating mastery (for example, selected 
response, constructed response, performance 
task, and personal communication). 


TLF IE2: TRACK STUDENT PROGRESS DATA 




Teacher: 1) routinely records the student progress 
data gathered in IE 1; 2) uses a system (for example, 
gradebooks, spreadsheets, charts) that allows for easy 
analysis of student progress toward mastery; and 3) at 
least half of the students know their progress toward 
mastery. 


Teacher: 1) routinely records the student 
progress data gathered in IE 1; and 2) 
uses a system (for example, gradebooks, 
spreadsheets, charts) that allows for easy 
analysis of student progress toward mastery. 


TLF IE3: IMPROVE PRACTICE AND RE-TEACH IN RESPONSE TO DATA 


n 


In response to IE 2, the teacher: 1) re-teaches, as 
appropriate; 2) modifies long-term plans, as appropriate; 
and 3) modifies practice, as appropriate. 


In response to IE 2, the teacher: 1) re-teaches, 
as appropriate; and 2) modifies long-term 
plans, as appropriate. 



Each line of the rubric is assessed independently 
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LEVEL 2 LEVEL 1 (LOWEST) 



\ 





Teacher routinely uses assessments to measure 
student mastery of content standards. 


Teacher does not routinely use assessments to measure 
student mastery of content standards. 




Teacher routinely records the student progress data 
gathered in IE 1. 


Teacher does not routinely record student progress 
data gathered in IE 1. 




In response to IE 2, the teacher re-teaches, as 
appropriate. 


Teacher does not re-teach. 

J 
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DISTRICT OF COLUMBIA 

PUBLIC SCHOOLS 

THE MISSION OF THE DISTRICT OF COLUMBIA PUBLIC SCHOOLS IS TO ENSURE THAT ALL 
STUDENTS ACQUIRE THE KNOWLEDGE, SKILLS AND VALUES NECESSARY TO LIVE RICH AND 
FULFILLING LIVES AS RESPONSIBLE, PRODUCTIVE AND ENLIGHTENED MEMBERS OF A DEMO- 
CRATIC SOCIETY. THE DISTRICT OF COLUMBIA PUBLIC SCHOOLS WILL BE KNOWN AS ONE OF 
THE BEST URBAN SCHOOL DISTRICTS IN THIS COUNTRY. THE DISTRICT OF COLUMBIA PUBLIC 
SCHOOLS WILL OFFER AN OUTSTANDING EDUCATION TO EVERY STUDENT WITHIN A SAFE, 
HEALTHY AND EDUCATIONALLY APPROPRIATE ENVIRONMENT. THE DISTRICT OF COLUMBIA 
PUBLIC SCHOOLS WILL BE AMONG THE FIRST MAJOR URBAN SCHOOL DISTRICTS TO ELIMI- 
NATE THE ACHIEVEMENT GAP AMONG ALL SUBGROUPS OF OUR STUDENT POPULATION. THE 
DISTRICT OF COLUMBIA PUBLIC SCHOOLS WILL DYNAMICALLY ENGAGE PARENTS AND THE 
COMMUNITY IN THE LIVES OF OUR STUDENTS AND SCHOOLS. THE DISTRICT OF COLUMBIA 
PUBLIC SCHOOLS WILL BE THE FIRST AND BEST CHOICE FOR FAMILIES LIVING IN THE DISTRICT 
OF COLUMBIA. 
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THE MISSION OFTHE ASPEN INSTITUTE ISTO FOSTER ENLIGHTENED LEADERSHIP AND OPEN- 
MINDED DIALOGUE. THROUGH SEMINARS, POLICY PROGRAMS, CONFERENCES, AND LEAD- 
ERSHIP DEVELOPMENT INITIATIVES, THE INSTITUTE AND ITS INTERNATIONAL PARTNERS SEEK 
TO PROMOTE NONPARTISAN INQUIRY AND AN APPRECIATION FOR TIME-LESS VALUES. THE 
INSTITUTE IS HEADQUARTERED IN WASHINGTON, D.C., AND HAS CAMPUSES IN ASPEN, COLO- 
RADO, AND ON THE WYE RIVER ON MARYLAND'S EASTERN SHORE. ITS INTERNATIONAL NET- 
WORK INCLUDES PARTNER ASPEN INSTITUTES IN BERLIN, ROME, LYON, TOKYO, AND NEW 
DELHI, AND LEADERSHIP PROGRAMS IN AFRICA AND CENTRAL AMERICA. 



THE ASPEN ) INSTITUTE 

EDUCATION & SOCIETY PROGRAM 

THE EDUCATION AND SOCIETY PROGRAM PROVIDES AN INFORMED AND NEUTRAL FORUM 
FOR EDUCATION PRACTITIONERS, RESEARCHERS, AND POLICY LEADERS TO ENGAGE IN FO- 
CUSED DIALOGUE REGARDING THEIR EFFORTS TO IMPROVE STUDENT ACHIEVEMENT, AND 
TO CONSIDER HOW PUBLIC POLICY CHANGES CAN AFFECT PROGRESS. THROUGH OUR MEET- 
INGS, ANALYSIS, COMMISSIONED WORK, AND STRUCTURED NETWORKS OF POLICYMAKERS 
AND PRACTITIONERS, THE PROGRAM FOR NEARLY 30 YEARS HAS DEVELOPED INTELLECTUAL 
FRAME-WORKS ON CRITICAL EDUCATION ISSUES THAT ASSIST FEDERAL, STATE, AND LOCAL 
POLICYMAKERS WORKING TO IMPROVE AMERICAN EDUCATION. 
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